PALEVO 1631-0683 Elsevier S1631-0683(10)00070-9 10.1016/j.crpv.2010.07.002 Research article General palaeontology 3D geometric morphometrics and missing-data. Can extant taxa give clues for the analysis of fossil primates? Morphométrie géométrique 3D et données manquantes. Les taxons actuels peuvent-ils nous donner des indications pour étudier les primates fossiles ? Clément Gaël Geffard-Kuriyama Didier Couette Sébastien couette@mnhn.fr a White Jess b UMR CNRS 7207 “CR2P”, département histoire de la Terre, école pratique des hautes études, Muséum national d’histoire naturelle, CP 38, 8, rue Buffon, 75005 Paris, France Department of Sociology and Anthropology, Functional Morphology and Evolutionary Anatomy Working Group, 404 Morgan Hall, Western Illinois University, Macomb, Illinois 61455 USA 9 6-7 S1631-0683(10)X0006-9 Imaging & 3D in palaeontology and palaeoanthropology 3D & imagerie en sciences paléontologiques et paléoanthropologiques 423 433 © 2010 Académie des sciences. Published by Elsevier B.V. All rights reserved. 2010 Académie des sciences Full (PDF)

Geometric morphometric methods constitute a powerful and precise tool for the quantification of morphological differences. The use of geometric morphometrics in palaeontology is very often limited by missing data. Shape analysis methods based on landmarks are very sensible but until now have not been adapted to this kind of dataset. To analyze the prospective utility of this method for fossil taxa, we propose a model based on prosimian cranial morphology in which we test two methods of missing data reconstruction. These consist of generating missing-data in a dataset (by increments of five percent) and estimating missing data using two multivariate methods. Estimates were found to constitute a useful tool for the analysis of partial datasets (to a certain extent). These results are promising for future studies of morphological variation in fossil taxa.

Les méthodes de morphométrie géométrique constituent un outil puissant et précis dans la quantification des différences morphologiques. Cependant, l’application des méthodes de morphométrie géométriques en paléontologie soulève le problème des données manquantes. Le matériel étant souvent fragmentaire, les méthodes d’analyse de forme, et notamment les méthodes faisant appel aux points homologues, sont inadaptées à ce type de données. Dans une perspective d’application aux fossiles, un modèle de test des méthodes de reconstruction des données manquantes est proposé sur un échantillon de primates prosimiens. Ce modèle consiste à générer des données manquantes à partir d’un jeu de données complet (par tranche de 5 %) et de reconstruire ces données manquantes. La pertinence des reconstructions est testée. Les résultats indiquent que, dans une certaine limite, les méthodes de reconstruction permettent d’inclure dans l’analyse les spécimens dont la préservation est partielle. Ces conclusions sont prometteuses pour l’analyse de la variation morphologique des taxons fossiles.

 Missing-data, Morphometrics, Estimation, Landmarks Données manquantes, Morphométrie, Estimation, Points homologues miscellaneous Written on invitation of the Editorial Board Introduction

Fossilization and preservation of organisms through time usually involve working with datasets that are, by the nature of palaeontology, incomplete. Multivariate morphometrics generally requires the use of a relatively complete dataset, therefore, using incomplete palaeontological collections can present a real methodological dilemma. This is especially true in the case of geometric morphometrics, in which variables are not only used in describing the dimensions or shapes of specimens, but are parts of the morphology itself (e.g. coordinates of landmarks). With this methodological complication in mind, palaeontologists commonly choose to either work on extant data, select a subsection of data that may be present on all of the available specimens, or work only on complete specimens, thereby excluding from the sample specimens with missing data. In any scenario, the palaeontologist is then presented with either choosing to exclude a part of the morphology from any analysis or decreasing the sample size for analysis. While both of these situations may be offset by working with large samples, in the case of small samples, these solutions may prove too restrictive or unworkable.

Recently, a great deal of literature has been devoted to the issue of estimating missing-data in an incomplete dataset (e.g. Holt and Benfer, 1994, Holt and Benfer, 2000, Motani, 1997, Neeser et al., 2009, Petersen et al., 2006, Schafer, 1997, Strauss and Atanassov, 2006 and Strauss et al., 2003). Various techniques are available, ranging from the intuitive visual estimation, more complicated processes of multivariate computation, to even three-dimensional virtual reconstruction (Zollikofer and Ponce de León, 2005). Estimation of missing data is often criticized by some palaeontologists who argue these methods could introduce bias into the estimation of variances. This is particularly the case when missing-data are estimated from means or average values of specimens with landmarks or measurements that are observable, or by simple regression of a variable onto another (Little, 1992 and Little and Rubin, 1987). While some have suggested methods to correct for these errors, there is still debate regarding their utility (e.g. Snedecor and Cochrane, 1989, Sokal and Rohlf, 1995 and Steel and Torrie, 1980). For example, maximum likelihood-based techniques (essentially Expectation Maximization [EM]) (Dempster et al., 1977) and Multiple Imputation [MI] (Schafer and Olsen, 1998) processes have been suggested to be the most effective. However, these methods were applied to unique datasets, and the lack of applicability of these methods to other datasets may prohibit their potential utility. For example, each method has assumptions that must be verified, such as multivariate normality or random distribution of missing values. Such assumptions are very difficult to verify on biological or palaeontological datasets. Geometric morphometric data also have their own statistical particularities; for example, X, Y and Z coordinates must be collinear and the position of one landmark is necessarily linked to the position of others on a specimen. Another restriction with any palaeontological dataset is the sample size, which can be very small. Thus, it may be inappropriate to use some methods, which are statistically restricted on some datasets. Considering all these parameters, functional morphologists continue to struggle with theoretical approaches to missing-data estimation methods, particularly methods involving landmark coordinates.

In the analysis of fossil material, reconstruction methods commonly rely on a single, or several reference specimens. Using this model, the shapes of incomplete specimens are fitted onto the known morphology of an undamaged specimen, and missing coordinates are estimated (Gunz, 2005, Gunz et al., 2004, Gunz et al., 2009, Neeser et al., 2009, Ponce de León, 2002, Ponce de León and Zollikofer, 1999 and Zollikofer and Ponce de León, 2005). Although various methods have been proposed, the reference specimen is absolutely necessary. When it is difficult to identify an appropriate reference specimen, an average specimen can be computed for use in any subsequent data estimation (see appendix in Cardini and Elton (2008)). By its nature, the definition of a reference specimen on which to base all other estimations can be problematic. For example, the correct taxonomic affiliation must be known, the sex must be identified, potential sexual dimorphism must be taken into account, and low intra-taxon variation must be assumed. These a priori assumptions all constitute substantial difficulties when working with any palaeontological dataset.

Godinot and Couette (2008) conducted a taxonomic review of the large adapines, a group of European fossil primates, based on dental and cranial morphology. In this analysis, preliminary morphometric study was performed on a sample of adapine crania, both complete and damaged. In order to include damaged crania, the authors visually estimated the locations of the missing landmarks. Whereas the measurement error was low in the analysis (attesting to the estimation quality), the authors recognized that this approach is highly subjective, requiring constant reevaluation in future studies. With this result acting as a framework for further analysis, here we test the accuracy of two estimation methods on a sample of extant primates by both generating and reconstructing missing-data. By doing so, we aim to test the reliability of two different methods of estimating missing-data: Multiple Regression (MR) and Expectation Maximization (EM).

The aim of the current study is not to study morphological variation in fossil primates; rather, we anticipate that our testing of the power of reconstruction methods can eventually be applied, in the future, to a sample of extinct primates. By using a set of extant prosimian primates with a complete set of landmarks to test the power of missing data estimation, we ask the following questions: (1) is there a maximum limit to the amount of missing-data that can be estimated; (2) can we rely on methods of reconstructing missing data to study morphological differences using geometric morphometrics; and (3) is it ultimately of benefit to reconstruct missing-landmarks? We anticipate that, by addressing these research questions, we will gain new insight into the use of reconstructing missing data in any future analysis of fossil primates.

 Materials and methods

Data were collected on two genera (five species, total) in the collections of Laboratoire mammifères et oiseaux of the Muséum national d’histoire naturelle, Paris, France (n = 86): Hapalemur griseus (N = 10), Eulemur coronatus (N = 10), Eulemur macaco (N = 10), Eulemur mongoz (N = 8), Eulemur fulvus albocollaris (N = 10), Eulemur fulvus albifrons (N = 10), Eulemur fulvus fulvus (N = 10), Eulemur fulvus mayottensis (N = 10), Eulemur fulvus rufus (N = 8). The sample was solely composed of adult specimens (specimens that demonstrated eruption of the entire adult dentition).

A total of 80 landmarks were defined on the crania and digitized using a Microscribe G2X digitizer (Immersion Corporation, San Jose, California). Landmarks were located on the sagittal plane of each side of the cranium (Fig. 1 and Table 1). In order to compute a measurement error, each specimen was digitized twice. The resulting rate of measurement error varied from 2.3 to 2.7%; with such a low rate, we interpret that there is a negligible effect on the results presented below. The effect of sex on cranial variation (sexual dimorphism) was also tested using a GLM procedure and rejected.

 Missing-data estimation

The process we took in exploring the effect of missing data on geometric morphometric analysis was composed of several steps. Firstly, the entire dataset was subjected to random deletion of landmarks in 5% increments using the R software (2008). As landmarks coordinates are defined by X, Y and Z, the procedure of random deletion was performed only on the X coordinates with the Y and Z matrices manually adjusted accordingly (to maintain homogeneity and coherence between X, Y and Z coordinates). Secondly, landmarks that were present on one side, but missing on the other, were mirrored on each specimen (and repeated for each 5% iteration of the deletion process). The resulting dataset was then comprised of landmarks that had been digitized in the original data capture procedure, landmarks that were mirrored from the opposite side of the crania, and missing-landmarks. To check for bias, values of error (based upon the deviation of the midline points from the midline plane) were calculated. These errors were negligible. Finally, missing landmarks (X, Y, and Z components) were estimated using one of two methods – Multiple Regression (MR) and Expectation Maximization (EM). The EM procedure is an iterative method. It consists on an initial estimation of missing values by substituting the means variables by variable, computing a set of parameters (means, variances, covariances…), re-estimating the missing values from this set of parameters, estimating these parameters, until the parameters converge on a final value (see Dempster et al. (1977) for more details). In this case, we used 1000 iterations or a convergence value of 0.001. The multiple regression technique consists on the estimation of a missing value on a specimen for a variable from the values of the other variables and other specimens (Little and Rubin, 1987 and Sokal and Rohlf, 1995). Both of these procedures have been performed taking taxonomic groups into account.

For each step of deletion, and estimation, an estimation error has been computed using the R software. The error was quantified by calculating the deviation between the original coordinates of specimens and the estimated X, Y, Z coordinates. The total estimation error for each step was calculated as the average value of errors computed on specimens using the estimated landmarks coordinates only (we did not include full specimens in the computation).

 Morphospace differentiation

We analyzed nine different datasets: the original dataset, datasets obtained after 5, 10, 15 and 20% missing-data estimations using EM and datasets obtained after 5, 10, 15 and 20% missing-data estimations using the MR method. For each dataset, we applied a generalized Procrustes analysis, using a Generalized Least-Squares (GLS) algorithm, to perform translation, rotation, and scaling (via the unit of centroid size). With this procedure, differences in shape are reported as residuals from each transformed landmark or as uniform changes in the overall shape (Rohlf and Marcus, 1993 and Rohlf and Slice, 1990). Following the Procrustes transformation, a Principal Component Analysis (PCA) was performed for each set of landmarks to build a set of morphospaces (multivariate statistical spaces where the position of a specimen characterizes its morphology). Following this, we estimated the variation of specimen locations, between the different morphospaces for each of both estimation methods. We also constructed visualizations of the cranial geometry using the Morphologika v 2.5. software (O’Higgins and Jones, 2006) to suggest potential biological interpretations resulting from each step of missing-data estimation.

 Morphological taxonomic differences

Discriminant Function Analysis (DFA) was applied to the different sets of landmarks for each of the two estimation methods to determine if the shape of genera, species and subspecies could be distinguished from others statistically. DFA was used in this context as it emphasizes relationships among group covariance matrices to discriminate between groups (see, among others, Morrison, 1990 and Pielou, 1984). Canonical Variate Analyzes (CVA) were performed on DFA functions and statistical tests (a posteriori statistics for classification and percent of correct classifications) computed. Both the DFA and CVA procedures were calculated using the R software (2008).

 Results Missing-data estimation

The impact of missing-data on the initial sample is presented in Fig. 2. Firstly, when a landmark was missing for one specimen of the sample, this landmark was excluded for all the specimens (black line). As some specimens do not necessarily share or exhibit particular landmarks, when data is excluded because it is missing in one specimen, the loss of information is dramatic. For example, the percentage of shared landmarks is reduced by more than 50% when 3% of the data in the sample is missing. The percentage is reduced to less than 25% when only 5% of the data are missing. The sample size is dramatically reduced when specimens with missing-data are excluded (Fig. 2, grey line). More than 90% of the specimens are excluded of the study if 3% of the landmarks are missing.

In general, the mirror reflection method of estimating missing data served as an efficient solution for the estimation of pair landmarks (where one landmark was present on one side, but missing on the other). By utilizing the mirror reflection method, we were able to considerably reduce the amount of missing data. After that, multivariate estimation methods are used. The percentage of estimation error as a function of percentage of missing data is shown in Fig. 3. The percentage of estimation error is computed after the reflecting procedure. Estimation error increases exponential with both EM and multiple regression methods. If we consider an empirical 10% threshold for estimation error, a limit value of 20% of missing data in the sample would be proposed by the model. In other words, after 20% of missing data in the sample, the error due to estimation is higher than 10%. Considering this result, we decided not to estimate more than 20% of missing-data.

 Morphospace occupation and visualizations

Fig. 4 illustrates the results along the first two principle axes after Principal Components Analysis on the Procrustes fitted coordinates following data estimation using both EM and MR methods. Using the full dataset (with no missing-data), the first principle component represented 80% of the total shape variation. With the deletion of data in 5% increments and using the EM model for data estimation, the total variation represented by the first component was as following: 5–73%, 10–61%, 15–52%, 20–47%. Thus, with an increasing amount of missing and estimated data, the total amount of variation represented by the first component decreased (a negative relationship). Using the MR method, the first principle component represented the following percentage of shape variation with increments of data deletion: 5–66%, 10–57%, 15–56%, and 20–38%. Compared with the EM method of estimation, a progressively lesser amount of shape variation could be explained by the first principle component at each data deletion level using the MR method, although the differences are not considerable. Using both methods of data estimation, the morphospace occupation increased with percentage of data deletion; in other words, the distinction between groups along the first principle component decreased. Results obtained here suggest that the potential for producing a larger number of outliers and extreme morphologies is slightly larger using the EM method than the MR method. On the other hand, the MR method of estimation would likely produce a greater level of morphological artifacts on the overall sample, rather than on particular specimens, than the MR method. The differences between the two methods could thus explain the differences observed along the first PC. We caution against overemphasizing these differences, however, as there are no substantial differences in the occupation of the morphospace illustrated in Fig. 4. It is notable that after visualization of extreme morphologies (outliers), it is evident that with levels of estimated data greater than 15%, some landmarks demonstrate aberrant locations and do not reflect any true biological condition.

 Morphological taxonomic differences

A DFA and a CVA on the discriminant functions were performed on the initial full dataset (with no missing-landmarks), with the first two canonical axes illustrated in Fig. 5. In this case, the first two canonical axes account for approximately 99% of the total shape variation. In this analysis, the cranial morphology of groups distinguished at the generic, specific, and subspecific levels were statistically distinguishable (in each comparison, the p- values of F statistics on Mahalanobis distances are all highly significant and a posteriori probabilities attest that all specimens are correctly classified in their groups). Following the analysis on the full dataset (with no missing landmarks), the same DFA and CVA analyses were performed using datasets with increasing data deletion and missing-data estimation using both the EM and MR methods (Fig. 6). In each case, the two first discriminant functions were significant (Chi2 test with a = 5%). Using EM method, the percentage of total variation along the first axis was as follows: 5–86%, 10–78%, 15–63%, and 20–51%. Using the MR method, the percentages of total variation along the first canonical axis were calculated as: 5–87%, 10–76%, 15–58%, and 20–46%.

Using both the EM and MR methods of data estimation, cranial morphologies are differentiated by the canonical functions at the 5 and 10% data deletion level (p-values of F statistics on Mahalanobis distances indicate significant difference). At the 5% data deletion level, 1% of the specimens are misclassified using the EM method, whereas 3% are misclassified using the MR method. At the 10% level, 7% are misclassified using EM, with 13% misclassified using the MR method. However, genera, species and subspecies exhibit significantly different morphologies after estimation of 10% of missing-data. Unlike at the 5 and 10% data deletion levels, discriminant functions could not separate the cranial morphologies at the subspecific level using the 15% data deletion dataset. At this step, species and genera morphologies remain significantly different. These results are similar with EM and MR methods, with 24 and 27% misclassification, respectively. After estimation of 20% of missing-data, species are not distinguishable. At this level of data deletion, only generic morphologies remain significantly different. In terms of misclassification of estimated data, at the 20% data deletion level, 47% of the observations are misclassified using the EM estimation method, with 51% using MR method.

 Discussion

Following simulated data deletion and landmark estimation as described here, we conclude that the impact of missing-data on 3D morphometric analysis is high, particularly as the amount of missing-data increases. Thus, it is clear that the investigator must be cautious in the estimation process as the choice of sampled specimens, deleted landmarks, or estimation methods may prove to be too restrictive. We concede, however, that palaeontologists are generally not in the procedural context presented here. Missing-data are rarely randomly distributed; instead, missing-landmarks are commonly located on the more fragile parts of a fossil. For example, considering primate crania, the zygomatic arches or the bones of the neurocranium are, in our experience, more often damaged when compared to portions of the face. Considering this, our results must be weighed against the distinct possibility that missing-data occur more often on particular portions of the skull, rather than in a random pattern. Our results concur with those discussed by Neeser et al. (2009). The number and definition of the landmarks themselves could influence the estimation. We did not assume any weight to the landmarks in our study but the homology (as defined by Bookstein (1991)) and the location of each landmark could be influenced by a peculiar cranial function or structure. We can assume that the presence or absence of one or another landmark (depending on the definitions and the locations of these landmarks) in the dataset would have different effects. Solutions to these problems could come from insight gained through investigating the effect of phenotypic integration and modularity, based on landmark correlations and covariations (e.g., Ackermann and Cheverud, 2000, Cheverud, 1995 and Marroig and Cheverud, 2001). A good knowledge of interactions between biological parts, as well as effect of functions, development, evolutionary history, etc., on morphologies would constitute a helpful element. It would improve the dataset composition to better fit the aim of the study.

Our results suggest that missing-landmarks cannot be reliably estimated after the 20% data deletion level. Once this level of data deletion has been reached, we find that the estimation error exceeds the 10% estimation error threshold. Strauss et al. (2003) suggested that almost 50% of missing-data can be estimated with accuracy using a small number of characters; our results also suggest that the number of landmarks could have an impact on the power of the estimation methods.

While using a mirror reflection method appears to serve as a relatively powerful and accurate solution for estimating missing-data, it does reduce the asymmetric variation between right and left sides of one specimen. This variation remains of high interest for studying the influence of environmental adaption on the development of fluctuating asymmetry (random variations between right and left sides) as a measurable expression of developmental instability (Klingenberg, 2003a, Klingenberg, 2003b, Klingenberg and Nijhout, 1999, Klingenberg et al., 1998, Leamy and Klingenberg, 2005 and Willmore et al., 2005). Thus, the study of asymmetry is a valuable tool for understanding processes that may lead to the covariation of traits when considering phenotypic integration and modularity concepts (e.g., Breuker et al., 2007, Klingenberg, 2003a, Klingenberg, 2003b, Klingenberg, 2004, Klingenberg, 2005, Klingenberg, 2008, Klingenberg, 2009 and Klingenberg et al., 2003). For this particular topic, then, a mirror reflection method would not be appropriate.

In 2009, Neeser and colleagues investigated a mean substitution method for estimating missing-landmarks. The mean substitution method is one based on substitution using Thin Plate Spline and multivariate regression techniques. These authors utilized three large sample units (n = 67, 107, and 628) with 29 cranial landmarks identified. Whereas Neeser et al. (2009) found differences in the accuracy of the estimation methods they tested, we found that our chosen methods (EM and MR) produce similar results. In comparison, Strauss et al. (2003) tested two methods (EM and Principal-Component) for morphometric missing-data estimation, concluding that the results were similar using each; this conclusion is similar to ours.

With their results in mind, Neeser et al. (2009) presented some concluding recommendations for estimating missing-landmarks, advising the use of reference-specimens or reference-samples. These reference-samples should, they concluded, be composed of specimens that are taxonomically close to the target research species (sister-groups, for example). These authors also suggested that the larger the reference sample is, the more accurate the resulting estimation. These authors conclude that the choice of an estimation method must be made after consideration of the size of the reference sample, as well as the preservation of the studied specimen. These results are not directly comparable to those obtained in our study, as we did not assume any reference specimens. Instead, we based our investigation on our own ultimate goal: using data estimation methods for palaeontological purposes. In fact, we considered the fact that the fossil primate sample studied by Godinot and Couette (2008) would not have necessarily fit with any extant prosimian reference sample model (especially due to the small sample size of the fossil sample). Several researchers have suggested that the affinities between adapines and extant prosimians warrant comparison between the two groups. These studies, however, produce varied results. For example, Gregory (1920) found similarities between the postcrania of Adapis and Lepilemur. Gingerich, 1975 and Gingerich, 1977, on the other hand, found Adapis to be more similar to Hapalemur, based on other regions of the postcrania. Grandidier (1905) also identified postcranial characters that suggest an affinity between Adapis and Cheirogaleus and Lemur. More recently, Schwartz and Tattersall, 1979 and Schwartz and Tattersall, 1985 supported affinities between adapines, Lepilemur and Hapalemur, whereas Beard et al. (1988) proposed that the wrist morphology of Adapis showed the greatest similarity to extant lemurs and lorises. Thus, while the choice of a lemuriform reference-sample to estimate missing-data of adapines is appropriate, it is not clear as to which species (or subspecies) of lemur would be most accurate. We suggest that the concept of analogy, rather than phylogenetic affinity, is also an appropriate starting point for working on the estimation of fossil missing-landmarks. For example, our choice of Hapalemur and Eulemur in the present study was used because the crania (and, accordingly, the landmark locations) of both genera strongly resemble those of Adapis, Leptadapis, and Magnadapis. Thus, we suggest that a comparison of adapine cranial morphology with these prosimians genera would likely be more appropriate than the one performed with the New World species Alouatta by Godinot and Couette (2008). It must be noted, however, that the goal of the latter study was also to investigate sexual dimorphism in the adapines, which is relatively low in extant prosimians. Hence, it is very clear that the initial question of a study would determine the dataset used for missing-data estimation. That way, depending on the goal of the studies, the sample could reflect different a priori hypotheses (e.g. phylogenetic scenarios, morphofunctional aspects or intra/interspecific variations). Missing-data estimation would be highly influenced by the sample choice, which must be considered even more cautiously.

With reference to our initial research questions, our results suggest that a level of data deletion greater than 15% (or possibly extended to 20%) serves as an upper-limit to the utility of data estimation. In this sample, data deletion levels greater than 15% produced relatively unreliable results. Under this maximum limit, however, methods of missing-data estimation have the potential to be very useful to study morphological differences using geometric morphometric techniques. We feel this is particularly true if the comparisons are between taxonomic groups. We conclude, therefore, that the estimation of missing-data constitutes an appropriate solution for palaeontological studies that include damaged specimens, or with comparisons with small sample sizes. Once again, however, we stress that any estimation process must be chosen with the purpose of the comparison in mind and with consideration of the amount of missing-data.

 Acknowledgements

The authors want to thank Jacques Cuisin and Julie Villemain for the access to the specimens, Rémi Laffont and Eloïse Zoukouba for their methodological advices. They also want to thank Gaël Clément and Didier Geffard-Kuriyama for inviting them to submit the article for this special volume. Finally, they would like to thank the GDR 2474 CNRS “Morphométrie et évolution des formes” which organized the VIth SMEF in May 2009 where the preliminary results of this study were initially presented and discussed.

 Ackermann R.R. Cheverud J.M. Phenotypic covariance structure in tamarins (genus: Saguinus): a comparison of variation patterns using matrix correlation and common principal component analysis Am. J. Phys. Anthropol. 111 2000 489–501 Beard K.C. Dagosto M. Gebo D.L. Godinot M. Interrelationships among primate higher taxa Nature 331 1988 712–714 Bookstein F.L. Morphometric tools for landmarks data 1991 Cambridge University Press Cambridge, Massachusetts Breuker C.J. Gibbs M. Van Dyck H. Brakefield P.M. Klingenberg C.P. Van Dongen S. Integration of wings and their eyespots in the speckled wood butterfly Pararge aegeria J. Exp. Zool. Part B 308 2007 454–463 Cardini A. Elton S. Does the skull carry a phylogenetic signal? Evolution and modularity in the guenons Biol. J. Linn. Soc. 93 2008 813–834 Cheverud J.M. Morphological integration in the saddle-back tamarin (Saguinus fuscicollis) cranium Am. Nat. 145 1995 63–89 Dempster A.P. Laird N.M. Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm J. R. Stat. Soc. Ser. B-Stat. Methodol. 39 1977 1–38 Gingerich P.D. Dentition of Adapis parisiensis and the evolution of lemuriform primates Tatterall I. Sussman R.W. Lemur biology 1975 Plenum Press New York 65–80 Gingerich P.D. New species of Eocene primates and the phylogeny of European Adapidae Folia Primatol. 28 1977 60–80 Godinot, M., Couette, S., 2008. Morphological diversity in the skulls of large adapines (Primates, Adapiformes) and its systematic implications. In: Delson, E., MacPhee, R. (Eds.), Mammalian evolutionary morphology: a tribute to Frederick S. Szalay, Dordrecht, The Netherlands, Springer, pp. 285-313. Grandidier, G., 1905. Recherches sur les lémuriens disparus, et en particulier ceux qui vivaient à Madagascar. Nouvelles archives du Muséum, 4e Sér. 7, pp. 1–143. Gregory W.K. On the structure and relations of Notharctus, an American Eocene primate Mem. Amer. Mus. Nat. Hist. (n. s.) 3 1920 49–243 Gunz, P., 2005. Statistical and geometric reconstruction of hominid crania: reconstructing australopithecine ontogeny. Ph.D. Dissertation, University of Vienna, 214 p. Gunz, P., Mitteroecker, P., Bookstein, F.L., Weber, G.W., 2004. Computed aided reconstruction of incomplete human crania using statistical and geometrical estimation methods. In: BAR int., Enter the past: computer applications and quantitative methods in archaeology, Oxford, pp. 92–94. Gunz P. Mitteroecker P. Neubauer S. Weber G.W. Bookstein F.L. Principles for the virtual reconstruction of hominin crania J. Hum. Evol. 57 2009 48–62 Holt B. Benfer R.A. Jr. Iterative improvement of missing data estimates Am. J. Phys. Anthropol. S18 1994 109–110 Holt B. Benfer R.A. Jr. Estimating missing data: an iterative regression approach J. Hum. Evol. 39 2000 289–296 Klingenberg C.P. A developmental perspective on developmental instability: theory, models and mechanisms Polak M. Developmental instability: causes and consequences 2003 Oxford University Press New York 14–34 Klingenberg C.P. Developmental instability as a research tool: using patterns of fluctuating asymmetry to infer the developmental origins of morphological integration Polak M. Developmental instability: causes and consequences 2003 Oxford University Press New York 427–442 Klingenberg C.P. Integration, modules and development: molecules to morphology to evolution Pigliucci M. Preston K. Phenotypic integration: studying the ecology and evolution of complex phenotypes 2004 Oxford University Press New York 213–230 Klingenberg C.P. Developmental constraints, modules and evolvability Hallgrímsson B. Hall B.K. Variation 2005 Academic Press San Diego 219–247 Klingenberg C.P. Morphological integration and developmental modularity Annu. Rev. Ecol. Evol. Syst. 39 2008 115–132 Klingenberg C.P. Morphometric integration and modularity in configurations of landmarks: tools for evaluating a priori hypotheses Evol. Dev. 11 2009 405–421 Klingenberg C.P. Nijhout H.F. Genetics of fluctuating asymmetry: a developmental model of developmental instability Evolution 53 1999 358–375 Klingenberg C.P. McIntyre G.S. Zaklan S.D. Left-right asymmetry of fly wings and the evolution of body axes Proc. R. Soc. B-Biol. Sci. 265 1998 1255–1259 Klingenberg C.P. Mebus K. Auffray J.-C. Developmental integration in a complex morphological structure: how distinct are the modules in the mouse mandible? Evol. Dev. 5 2003 522–531 Leamy L.J. Klingenberg C.P. The genetics and evolution of fluctuating asymmetry Annu. Rev. Ecol. Evol. Syst. 36 2005 1–21 Little R.J.A. Regression with missing X's: a review J. Am. Stat. Assoc. 87 1992 1227–1237 Little R.J.A. Rubin D.B. Statistical analysis with missing data 1987 Wiley New York 278 p Marroig G. Cheverud J.M. A comparison of phenotypic variation and covariation patterns and the role of phylogeny, ecology, and ontogeny during cranial evolution of new world monkeys Evolution 55 2001 2576–2600 Morrison D.F. Multivariate statistical methods 4th ed. 1990 Duxbury Press 498 p Motani R. New techniques for retrodeforming tectonically deformed fossils, with an example for ichthyosaurian specimens Lethaia 30 1997 221–228 Neeser E. Ackermann R.R. Gain J. Comparing the accuracy and precision of three techniques used for estimating missing landmarks when reconstructing fossil hominin crania Am. J. Phys. Anthropol. 140 2009 1–18 O’Higgins, P., Jones, A., 2006. Morphologika 2.5. Tools for shape analysis. York: Hull York Medical School, University of York. Available at http://www.york.ac.uk/res/fme/. Petersen H.C. Ahlstöm T. Vach W. Geometric morphometry, missing values, and multiple imputations Am. J. Phys. Anthropol. S42 2006 91 Pielou E.C. The interpretation of ecological data: a primer on classification and ordination 1984 Wiley New York p. 263 Ponce de León M.S. Computerized paleoanthropology and Neanderthals: the case of Le Moustier 1 Evol. Anthropol. 11 2002 68–72 Ponce de León M.S. Zollikofer C.P.E. New evidence from Le Moustier 1: computer-assisted reconstruction and morphometry of the skull Anat. Rec. 254 1999 474–489 R Development Core Team (2008). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL: http://www.R-project.org. Rohlf F.J. Marcus L.F. A revolution in morphometrics Trends Ecol. Evol. 8 1993 129–132 Rohlf F.J. Slice D.E. Extensions of the Procrustes method for the optimal superimposition of landmarks Syst. Zool. 39 1990 40–59 Schafer J.L. Analysis of incomplete multivariate data 1997 Chapman and Hall London p. 444 Schafer J.L. Olsen M.K. Multiple imputation for multivariate missing-data problems: a data analyst's perspective Multivariate Behav. Res. 33 1998 545–571 Schwartz J.H. Tattersall I. The phylogenetic relationships of Adapidae (primates, lemuriformes) Anthrop. Papers Amer. Mus. Nat. Hist. 55 1979 271–283 Schwartz J.H. Tattersall I. Evolutionary relationships of living lemurs and lorises (Mammalia, Primates) and their potential affinities with European Eocene Adapidae Anthrop. Papers Amer. Mus. Nat. Hist. 60 1985 1–100 Snedecor G.W. Cochrane W.G. Statistical methods 8th ed. 1989 Iowa State Univ. Press Ames, Iowa p. 593 Sokal R.R. Rohlf F.J. Biometry 3rd ed. 1995 W.H. Freeman New York 887 p Steel R.G.D. Torrie J.H. Principles and procedures of statistics 2nd ed. 1980 McGraw-Hill New York 633 p Strauss R.E. Atanassov M.N. Determining best complete subsets and specimens and characters for multivariate morphometric studies in the presence of large amounts of missing data Biol. J. Linn Soc. 88 2006 309–328 Strauss R.E. Atanassov M.N. Alves de Oliveira J. Evaluation of the principle-component and expectation-maximization methods for estimation of missing data in morphometric studies J. Vert. Pal. 23 2003 284–296 Willmore K.E. Klingenberg C.P. Hallgrímsson B. The relationship between fluctuating asymmetry and environmental variance in rhesus macaque skulls Evolution 59 2005 898–909 Zollikofer C.P.E. Ponce de León M.S. Virtual reconstruction: a primer in computed-assisted paleontology and biomedicine 2005 Wiley-Liss New York 333 p

Position of the landmarks on a crania of Eulemur fulvus. See Table 1 for definition. Picture for the Digimorph website (http://digimorph.org). Scale bar represents 1 cm.

Position des points homologues sur un crâne d’Eulemur fulvus. Se référer au Tableau 1 pour la définition des points. Images provenant du site Internet Digimorph (http://digimorph.org). L’échelle représente 1 cm.

Estimation of the missing-data impact on the sample. X axis represents the percentage of missing-data (landmarks). In case A (black line), Y axis is the percentage of landmarks shared by all the specimens. In case B (gray line), Y axis is the percentage of full specimens.

Estimation de l’impact des données manquantes sur l’échantillon. L’axe des X représente le pourcentage de données manquantes (points homologues). Dans le cas A (courbe noire), l’axe des Y représente le pourcentage de points homologues partagés par tous les spécimens. Dans le cas B (courbe grise), l’axe des Y représente le pourcentage de spécimens complets.

Percentage of estimation error as a function of percentage of missing-data in the sample for EM (black curve) and Multiple regression (gray curve) methods. Dotted line represents the 10% threshold.

Pourcentage d’erreur d’estimation, en fonction du pourcentage de données manquantes dans l’échantillon pour les méthodes EM (courbe noire) et régression multiple (courbe grise). La ligne pointillée indique le seuil d’erreur de 10 %.

Morphospace occupation of specimens from full dataset to estimation of 20% of missing-data using both EM and Multiple regression methods. The percentage for each scatter plot indicates the amount of estimated missing-data. For each case, a visualization of an “extreme” morphology (specimen indicated with a star) is given to control the biological meaning of the estimation.

Occupation de l’espace morphologique, depuis le jeu de données complet jusqu’à l’estimation de 20 % des données manquantes, pour les méthodes EM et régression multivariée. Le pourcentage dans chaque graphique indique la proportion de données manquantes. Une visualisation de la morphologie crânienne d’un individu « extrême » (figuré par une étoile) est donnée, afin de contrôler le sens biologique de l’estimation.

Canonical variate analysis performed on discriminant functions, grouping by taxon, of the full dataset (no missing-data).

Analyse canonique réalisée sur les fonctions discriminantes de l’échantillon complet (sans données manquantes) en regroupant par taxon.

Canonical variate analyzes performed on discriminant functions, grouping by taxon. Missing-data, from 5 to 20%, have been estimated using both EM and multivariate regression methods. See Fig. 5 for symbols legend.

Analyses canoniques réalisées sur les fonctions discriminantes, en regroupant par taxon. Les données manquantes, de 5 à 20 %, ont été estimées par les méthodes EM et de régression multivariée. Se référer à la Fig. 5 pour la légende des symboles.

Definition and position of the landmarks.

Définition et position des points homologues.

Label Position Definition Label Position Definition IS Midsagittal Interdentale PCON Both sides Posterior point of the occipital condyle BEN Midsagittal Base of the nasal aperture PAZ Both sides Anterior point of the zygomatic fossa NSL Midsagittal Nasal PPZ Both sides Posterior point of the zygomatic fossa NA Midsagittal Nasion PMN Both sides Premaxilla/Maxilla/Nasal suture BR Midsagittal Bregma PNL Both sides Maxilla/Nasal/Frontal suture LD Midsagittal Lambda LF Both sides Maxilla/Lacrymal/Frontal suture OPCR Midsagittal Opisitocranion LACR Both sides Lacrymal foramen OPI Midsagittal Opisthion SO Both sides Summit of the orbit BA Midsagittal Basion BO Both sides Base of the orbit BAB Midsagittal Basioccipital/Basisphenoid suture ZS Both sides Upper Zygomatic/Maxilla suture BABA Midsagittal Basisphenoid/Presphenoid suture ZI Both sides Lower Zygomatic/Maxilla suture PNS Midsagittal Palatines suture FM Both sides Anterior Frontal/Zygomatic suture PMX Midsagittal Maxilla/Palatine suture FMP Both sides Posterior Frontal/Zygomatic suture MPMX Midsagittal Premaxilla/Maxilla suture PT Both sides Pterion PM Both sides Maxilla/Premaxilla suture on the face TSP Both sides Temporal/Sphenoid/Parietal suture P3 Both sides Disto vestibular point of P3/alveolus AS Both sides Asterion M1 Both sides Disto vestibular point of Ml/alveolus ZYG0S Both sides Upper Zygomatic/Parietal suture MT Both sides Disto vestibular point of M3/alveolus ZYGUP Both sides Upper point of the zygomatic arch APET Both sides Anterior petrous temporal ZYGDOW Both sides Lower point of the zygomatic arch TS Both sides Temporal/Sphenoidal junction at the petrous ZYGO Both sides Lower Zygomatic/Parietal suture AB Both sides Anterior point of the bulla EAM Both sides Anterior external auditory meatus PB Both sides Posterior point of the bulla PEAM Both sides Posterior external auditory meatus ACON Both sides Anterior point of the occipital condyle ISC Both sides Intersection of the parieral crestwith the Frontal/Parietal suture LCON Both sides Lateral point of the occipital condyle